Context Switching with Offload Processors

ABSTRACT

A method for context switching of multiple offload processors is disclosed. The method can include receiving network packets for processing through a memory bus connected socket, organizing the network packets into multiple sessions for processing, suspending processing of at least one session by reading a cache state of at least one of the offload processor into a context memory by operation of a scheduling circuit, with virtual memory locations and physical cache locations being aligned, and subsequently directing transfer of the cache state to at least one of the offload processors for processing by operation of the scheduling circuit.

PRIORITY CLAIMS

This application claims the benefit of U.S. Provisional PatentApplications 61/753,892 filed on Jan. 17, 2013, 61/753,895 filed on Jan.17, 2013, 61/753,899 filed on Jan. 17, 2013, 61/753,901 filed on Jan.17, 2013, 61/753,903 filed on Jan. 17, 2013, 61/753,904 filed on Jan.17, 2013, 61/753,906 filed on Jan. 17, 2013, 61/753,907 filed on Jan.17, 2013, and 61/753,910 filed on Jan. 17, 2013, the contents all ofwhich are incorporated by reference herein.

TECHNICAL FIELD

Described embodiments relate to deterministic context switching forcomputer systems that include a memory bus connected module with offloadprocessors.

BACKGROUND

Context switching (sometimes referred to as a process switch or a taskswitch) is the switching of a processor from execution of one process orthread to another. During a context switch the state (context) of aprocess is stored in memory so that execution can be resumed from thesame point at a later time. This enables multiple processes to share asingle processor and support a multitasking operating system. Commonly,a process is an executing or running instance of a program that can runin parallel and share an address space (i.e., a range of memorylocations) and other resources with their parent processes. A contextgenerally includes the contents of a processor's registers and programcounter at a specified time. An operating system can suspend theexecution of a first process and store the context for that process inmemory, while subsequently retrieving the context of a second processfrom memory and restoring it in the processor's registers. Afterterminating or suspending the second process, the context of the firstprocess can be reloaded, resuming execution of the first process.

However, context switching is computationally intensive. A contextswitch can require considerable processor time, which can be on theorder of nanoseconds for each of the tens or hundreds of contextswitches per second. Since modern processors can handle hundreds orthousands separate processes, the time devoted to context switching canrepresents a substantial cost to the system in terms of processor time.Improved context switching methods and systems can greatly improveoverall system performance and reduce hardware and power requirementsfor server or other data processing systems.

SUMMARY

This disclosure describes embodiments of systems, hardware and methodssuitable for context switching of processors in a system. Embodimentscan include multiple offload processors, each connected to a memory bus,with the respective offload processors each having an associated cachewith an associated cache state. A low latency memory can be connected tomultiple offload processors through the memory bus, and a schedulingcircuit can be used for directing storage of a cache state from at leastone of the respective offload processors into the low latency memory,and for later directing transfer over the memory bus of the cache stateto at least one of the respective offload processors. In certainembodiments, including those associated with the use of ARM architectureprocessors, the multiple offload processors can have an acceleratorcoherency port for accessing cache state with improved speed. In otherembodiments, a common module can support the offload processors, lowlatency memory, and scheduling circuit, with access for external networkpackets provided through a memory socket mediated connection, includingbut not limited to dual in line memory module (DIMM) sockets.

In some embodiments the associated cache state includes at least one of:a state of the processor registers saved in register save area,instructions in the pipeline being executed, stack pointer and programcounter, prefetched instructions and data waiting to be executed by thesession, and data written into the cache. The system can further includean operating system (OS) running on at least one the multiple offloadprocessors. The OS and scheduling circuit can cooperate to establishsession contexts that are physically contiguous in a cache. Sessioncolor, size, and starting physical address can be communicated to thescheduler circuit upon session initialization and a memory allocatorused to determine a starting address of each session, the number ofsessions allowable in the cache, and the number of locations wherein asession can be found for a given color.

According to embodiments, a cache state stored by one of the multipleoffload processors can be transferred to another offload processor. Inparticular applications, this can enable a scheduling circuit toprioritize processing of network packets in a first queue received overthe memory bus by stopping a first session associated with one of theoffload processors, storing the associated cache state, and initiatingprocessing of network packets held in a second queue.

Embodiment can also include a method for context switching of multipleoffload processors, each having an associated cache with an associatedcache state, and using a low latency memory connected to multipleoffload processors through the memory bus. The method includes directingstorage of a cache state via a bulk read from at least one of therespective offload processors into the low latency memory using ascheduling circuit, with any virtual and physical memory locations beingaligned. Subsequently, transfer is directed over the memory bus of thecache state to at least one of the respective offload processors forprocessing, with transfer being controlled by the scheduling circuit. Aswith the structure embodiments previously described, common module cansupport the offload processors, low latency memory, and schedulingcircuit, with access for external network packets provided through aDIMM or other memory socket connection.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1-0 shows a system having context switching according to anembodiment.

FIG. 1-1 is a diagram showing page collisions in a physically indexedcache without coloring.

FIG. 1-2 shows a virtually indexed cache.

FIG. 1-3 shows a virtual/physical aligned cache according to anembodiment.

FIGS. 2-0 to 2-3 show processor modules according to variousembodiments.

FIG. 2-4 shows a conventional dual-in-line memory module.

FIG. 2-5 shows a system according to another embodiment.

FIG. 3 shows a system with a memory bus connected offload processormodule having context switching capabilities, according to oneembodiment.

FIG. 4 is a flow diagram showing context switching operations accordingto one particular embodiment.

DETAILED DESCRIPTION

Various embodiments will now be described in detail with reference to anumber of drawings. The embodiments show modules, systems and methodsfor switching contexts in offload processors that are connected to asystem memory bus. Such offload processors can be in addition to anyhost processors connected to the system memory bus, and can operate ondata transferred over the system memory bus independent of any hostprocessors. In particular embodiments, offload processors can haveaccess to a low latency memory, which can enable rapid storage andretrieval of context data for rapid context switching. In veryparticular embodiments, processing modules can populate physical slotsfor connecting in-line memory modules (e.g., dual in line memory modules(DIMMs)) to a system memory bus.

FIG. 1-0 shows a system 100 according to one embodiment. A system 100can include one or more offload processors 118, a scheduler 116, and acontext memory 120. An offload processor 118 can include one or moreprocessor cores that operate in conjunction with a cache memory. In acontext switch operation, the context of a first processing task ofoffload processor 118 can be stored in context memory 120, and theoffload processor 118 can then undertake a new processing task.Subsequently, the stored context can be restored from the context memory120 to the offload processor 118, and the offload processor 118 canresume the first processing task. In particular embodiments, the storingand restoring of context data can include the transfer of data betweenthe cache of an offload processor 118 and the context memory 120.

A scheduler 116 can coordinate context switches of offload processors118 based on received processing requests. Accordingly, a scheduler 116can be informed of, or can have access to, the state of offloadprocessors 118 as well as the location of context data for the offloadprocessors 118. Context data locations can include locations inprocessor cache, as well as locations in context memory 120. A scheduler116 can also follow, or be updated with, a state of offload processors118.

As understood from above, a context memory 120 can store context data ofoffload processors 118 for subsequent retrieval. A context memory 120can be separate from cache memories of the offload processors. In someembodiments, a context memory 120 can be low latency memory as comparedto other memory in the system, to enable rapid context storage andretrieval. In some embodiments, a context memory 120 can store dataother than context data.

In the particular embodiment shown, offload processors 118, scheduler116, and context memory 120 can be part of a module 122 connected to amemory bus 124. Data and processing tasks for execution by offloadprocessors 118 can be received over memory bus 124. In some embodiments,transfers of context data between offload processors 118 and contextmemory 120 can occur over memory bus 124. However, in other embodiments,such transfers can occur over a different data path on the module 122.

Referring still to FIG. 1-0, in the very particular embodiment shown, amethod can further include a second switch 114, a memory controller 112,a host processor 110, an input/output (I/O) fabric 108, and a firstswitch 106. A second switch 114 can be included on module 122. Theparticular system 100 of FIG. 1-0 is directed to network packetprocessing scheduling and traffic management, but it is understood thatother embodiments can include context switching operations, as describedherein or equivalents, directed to other types of processing tasks.

In the particular embodiment of FIG. 1-0, a first switch 106 can receiveand/or transmit data packets 104 from data source 102. A data source 102can be any suitable source of packet data, including the Internet, anetwork cloud, inter- or intra-data center networks, cluster computers,rack systems, multiple or individual servers or personal computers, orthe like. Data can be packet or switch based, although in particularembodiments non-packet data is generally converted or encapsulated intopackets for ease of handling. The data packets typically have certaincharacteristics, including transport protocol number, source anddestination port numbers, or source and destination (Internet Protocol)IP addresses. The data packets can further have associated metadata thathelps in packet classification and management.

A switch 106 can be a virtual switch (an I/O device). A switch 106 caninclude, but is not limited to, devices compatible with peripheralcomponent interconnect (PCI) and/or PCI express (PCIe) devicesconnecting with host motherboard via PCI or PCIe bus 107. The switch 106can include a network interface controller (NIC), a host bus adapter, aconverged network adapter, or a switched or an asynchronous transfermode (ATM) network interface. In some embodiments, a switch 106 canemploy IO virtualization schemes such as a single root I/Ovirtualization (SR-IOV) interface to make a single network I/O deviceappear as multiple devices. SR-IOV permits separate access to resourcesamong various PCIe hardware functions by providing both physical controland virtual functions. In certain embodiments, the switch 106 cansupport OpenFlow or similar software defined networking to abstract outof the control plane. The control plane of the first virtual switchperforms functions such as route determination, target nodeidentification etc.

A switch 106 can be capable of examining network packets, and using itscontrol plane to create appropriate output ports for network packets.Based on route calculation for the network packets or data flowsassociated with the network packets, the forwarding plane of the switch106 can transfer the packets to an output interface. An output interfaceof the switch may be connected with an IO bus, and in certainembodiments the switch 106 may have the capability to directly (orindirectly, via an I/O fabric 108) transfer the network packets to amemory bus interconnect 109 for a memory read or write operation (directmemory access operation). Functionally, for certain applications thenetwork packets can be assigned for transport to specific memorylocations based on control plane functionality.

Switch 106, connected to an IO fabric 108 and memory bus interconnect109, can also be connected to host processor(s) 110. Host processor(s)110 can include one or more host processors which can providecomputational services including a provisioning agent 111. Theprovisioning agent 111 can be part of an operating system or user coderunning on the host processor(s) 110. The provisioning agent 111typically initializes and interacts with virtual function driversprovided by system 100. The virtual function driver can be responsiblefor providing the virtual address of the memory space where a directmemory addressing (DMA) is needed. Each device driver can be allocatedvirtual addresses that map to the physical addresses. A device model canbe used to create an emulation of a physical device for the hostprocessor 110 to recognize each of the multiple virtual functions (VF)that can be created. The device model can be replicated multiple timesto give the impression to VF drivers (a driver that interacts with avirtual IO device) that they are interacting with a physical device. Forexample, a certain device model may be used to emulate a network adapterthat the VF driver can act to connect. The device model and the VFdriver can be run in either privileged or non-privileged mode. There canbe no restriction with regard to which device hosts/runs the codecorresponding to the device model and the VF driver. The code, however,can have the capability to create multiple copies of device model and VFdriver so as to enable multiple copies of said I/O interface to becreated. In certain embodiments the operating system can also create adefined physical address space for applications supported by VF drivers.Further, the host operating system can allocate a virtual memory addressspace to the application or provisioning agent. The provisioning agent111 can broker with the host operating system to create a mappingbetween virtual addresses and a subset of the available physical addressspace. The provisioning agent 111 can be responsible for creating eachVF driver and allocating it a defined virtual address space.

A second virtual switch 114 can also be connected to the memorycontroller 112 using memory bus 109. The second virtual switch 114receives and switches traffic originating from the memory bus 109 bothto and from offload processors 118. Traffic may include, but is notlimited to, data flows to virtual devices created and assigned by theprovisioning agent 111, with processing supported by offload processors118. The forwarding plane of the second virtual switch transportspackets from a memory bus 109 to offload processors 118 or from theoffload processors 118 back onto the memory bus 109. For certainapplications, the described system architecture allows relatively directcommunication of network packets to the offload processors 118 withminimal or no interruptions to a host processor 110. The second virtualswitch 114 can be capable of receiving packets and classifying themprior to distribution to different hardware schedulers based on adefined arbitration and scheduling scheme. The hardware scheduler 116receives packets that can be assigned to flow sessions that arescheduled for processing in one or more separate session.

A scheduler 116 can control processing tasks for execution by offloadprocessors 118, including the switching of contexts. In someembodiments, metadata included within data received over memory bus 124(or metadata derived from such data) can be used to by scheduler 116 theschedule/switch tasks of the offload processors 118. However, commandbased control of a scheduler via memory bus received commands or flagsis also contemplated.

In the particular embodiment of FIG. 1-0, a scheduler 116 can beemployed to implement traffic management of incoming packets. Packetsfrom a certain source, relating to a certain traffic class, pertainingto a specific application or flowing to a certain socket are referred toas part of a session flow and are classified using session metadata.Session metadata often serve as the criterion by which packets areprioritized and as such, incoming packets can be reordered based ontheir session metadata. This reordering of packets can occur in one ormore buffers and can modify the traffic shape of these flows. Packets ofa session that are reordered based on session metadata can be sent overto specific traffic managed queues that are arbitrated out to outputports using an arbitration circuit (not shown). The arbitration circuitcan feed these packet flows to a downstream packetprocessing/terminating resource directly. Certain embodiments providefor integration of thread and queue management so as to enhance thethroughput of downstream resources handling termination of network datathrough above said threads.

Referring still to FIG. 1-0, data arriving at the scheduler 116 may alsobe packet data waiting to be terminated at the offload processors 118 orit could be packet data waiting to be processed, modified or switchedout. The scheduler 116 can be responsible for segregating incomingpackets into corresponding application sessions based on examination ofpacket data. The scheduler 116 can have circuits for packet inspectionand identifying relevant packet characteristics. In some embodiments, ascheduler 116 can offload part of the network stack to free offloadprocessors 118 from overhead incurred from network stack processing. Inparticular embodiments, a scheduler 116 can carry out any of:TCP/transport offload, encryption/decryption offload, segmentation andreassembly thus allowing offload processors to operate on payloads ofnetwork packets directly.

A scheduler 116 can further have the capability to transfer the packetsbelonging to a session into a particular traffic management queue. Ascheduler 116 can to control the scheduling of each of multiple suchsessions into a general purpose OS. The stickiness of sessions across apipeline of stages, including a general purpose OS, can be supported byscheduler 116 optimizing operations carried out at each of the stages.Particular embodiments of such operations are described in more detailbelow.

While a scheduler 116 have any suitable form, a scheduling circuit whichcan be used all, or in part, as a scheduler is shown in U.S. Pat. No.7,760,715, issued to Dalal on Jul. 20, 2010 (hereinafter the '715patent), and is incorporated herein by reference. The '751 patentdiscloses a scheduling circuit that takes account of downstreamexecution resources. The session flows in each of these queues is sentout through an output port to a downstream network element.

A scheduler can employ an arbitration circuit to mediate access ofmultiple traffic management output queues to available output ports.Each of the output ports may be connected to one of the offloadprocessor cores through a packet buffer. The packet buffer may furtherinclude a header pool and a packet body pool. The header pool may onlycontain the header of packets to be processed by offload processors 118.Sometimes, if the size of the packet to be processed is sufficientlysmall, the header pool may contain the entire packet. Packets can betransferred over to the header pool or packet body pool depending on thenature of operation carried out at the offload processor 118. For packetprocessing, overlay, analytics, filtering and such other applications itmight be appropriate to transfer only the packet header to the offloadprocessors 118. In this case, depending on the handling of the packetheader, the packet body might either be sewn together with a packetheader and transferred over an egress interface or dropped. Forapplications requiring the termination of packets, the entire body ofthe packet might be transferred. Offload processors can thus receive thepackets and execute suitable application session on them.

A scheduler 116 can schedule different sessions on the offloadprocessors 118, acting to coordinate such sessions to reduce theoverhead during context switches. A scheduler 116 can arbitrate, notjust between outgoing queues or session flows at line rate speeds, butalso between terminated sessions at very high speeds. A scheduler 116can manage the queuing of sessions on offload processors 118 and beresponsible for invoking new application sessions on the OS. A scheduler116 can indicate to the OS that packets for a new session are availablebased on traffic.

A scheduler 116 can also be informed of the state of the executionresources of offload processors 118, the current session that is run onthe execution resource and the memory space allocated to it, as well asthe location of the session context in the offload processor cache. Ascheduler 116 can thus use the state of the execution resource to carryout traffic management and arbitration decisions.

In the embodiment shown, a scheduler 116 can provide for an integrationof thread management on the operating system with traffic management ofincoming packets. It can induce a persistence of session flows across aspectrum of components including traffic management queues andprocessing entities on the offload processors 118. An OS running on anoffload processor 118 can allocate execution resources such as processorcycles and memory to a particular queue it is currently handling. The OScan further allocate a thread or a group of threads for that particularqueue, so that it can be handled distinctly by the general purposeprocessing element as a separate entity. Having multiple sessionsrunning on a general purpose (GP) processing resource (e.g., offloadprocessor resource), each handling data from a particular session flowresident in a queue on the scheduler 116, can tightly integrate thescheduler 116 and the GP processing resource. This can bring an elementof persistence within session information across the traffic managementand scheduler 116 and the GP processing resource.

In some embodiments, an offload processor 118 OS can be modified from aprevious OS to reduce the penalty and overhead associated with contextswitch between resources. This can be further exploited by the hardwarescheduler to carry out seamless switching between queues, andconsequently their execution as different sessions by the executionresource.

According to particular embodiments, a scheduler 116 can implementtraffic management of incoming packets. Packets from a certain source,relating to a certain traffic class, pertaining to a specificapplication or flowing to a certain socket are referred to as part of asession flow and are classified using session metadata. Session metadatacan serve as the criterion by which packets are prioritized and as such,incoming packets are reordered based on their session metadata. Thisreordering of packets can occur in one or more buffers and can modifythe traffic shape of these flows. Packets of a session that arereordered based on session metadata can be sent over to specific trafficmanaged queues that are arbitrated out to output ports using anarbitration circuit. An arbitration circuit can feed these packet flowsto a downstream packet processing and/or terminating resource directly(e.g., offload processor). Certain embodiments provide for integrationof thread and queue management so as to enhance the throughput ofdownstream resources handling termination of network data through abovethe threads.

In addition to carrying out traffic management, arbitration andscheduling of incoming network packets (and flows), a scheduler 116 canbe responsible for enabling minimal overhead context switching betweenterminated sessions on the OS offload processor 118. Switching betweenmultiple sessions on sessions of the offload processors 118 can makes itpossible to terminate multiple sessions at very high speeds. In theembodiment shown, rapid context switching can occur by operation ofcontext module 120. In particular embodiments, a context memory 120 canprovide for efficient, low latency context services in a system 100.

In the particular embodiment shown, packets can be transferred over tothe scheduler 116 by operation of second switch 114. Scheduler 116 canbe responsible both for switching between a session and a new session onthe offload processors 118, as well as initiating the saving of contextin context memory 120. The context of a session can include, but is notlimited to: the state of the processor registers saved in register savearea, the instructions in the pipeline being executed, stack pointer andprogram counter, prefetched instructions and data that are waiting to beexecuted by the session, data written into the cache recently and anyother relevant information that can identify a session executing on theoffload processor 118. In particular embodiments, a session context canbe identified using the combination of session id, session index in thecache, and starting physical address.

As will be described in more detail with respect to FIG. 1-3, atranslation scheme can be used such that contiguous pages of a sessionin virtual memory are physically contiguous in a cache of an offloadprocessor 118. This contiguous nature of the session in the cache canallow for a bulk read out of the session context into a ‘contextsnapshot’ for storage in context memory 120, from where it can beretrieved when the operating system (OS) switches processor resourcesback to the session. The ability to seamlessly fetch session contextfrom a context memory 120 (which can be a low latency memory, and thusbe orders of magnitude faster than a main memory of a system) can serveto effectively expand the size of an L2 cache of offload processors 118.

In some embodiments, an OS of system 100 can implement optimizations inits input output memory management unit (IOMMU) (not shown) to allow atranslation lookaside buffer (TLB) (or equivalent lookup structure) todistinctly identify contents of each session. Such an arrangement canallow address translations to be identified distinctly during a sessionswitch out and transferred to a page table cache that is external to theTLB. Use of a page table cache can allow for an expansion in the size ofthe TLB. Also given the fact that contiguous locations in the virtualmemory are at contiguous locations in physical memory and in aphysically indexed cache, the number of address translations requiredfor identifying a session are significantly reduced.

In the particular embodiment of FIG. 1-0, a system 100 can be wellsuited to provide out session and packet termination services. In someembodiments, control of network stack processing can be performed by ascheduler 116. Thus, a scheduler 116 can act as a traffic managementqueue, arbitration circuit and a network stack offload device. Thescheduler 116 can be responsible for handling entire session and flowmanagement on behalf of offload processors 118. In such an arrangement,offload processors 118 can be fed with the packets pertaining to asession directly into a buffer, from where it can extract packet datafor use. Processing of a network stack can be optimized to avoidswitches to a kernel mode to handle network generated interrupts (andexecute an interrupt service routine). In this way, a system 100 can beoptimized to carry out context switching of sessions seamlessly and withas little overhead as possible.

Referring still to FIG. 1-0, as will be understood, multiple types ofconventional input/output busses such as PCI, Fibre Channel can be usedin the described system 100. The bus architecture can also be based onrelevant JEDEC standards, on DIMM data transfer protocols, onHyperTransport, or any other high speed, low latency interconnectionsystem. Offload processors 118 may include DDR DRAM, RLDRAM, embeddedDRAM, next generation stacked memory such as Hybrid Memory Cube (HMC),flash, or other suitable memory, separate logic or bus management chips,programmable units such as field programmable gate arrays (FPGAs),custom designed application specific integrated circuits (ASICs) and anenergy efficient, general purpose processor such as those based on ARM,ARC, Tensilica, MIPS, Strong/ARM, or RISC architectures. Host processors110 can be a general purpose processor, including those based on Intelor AMD x86 architecture, Intel Itanium architecture, MIPS architecture,SPARC architecture or the like.

As will also be understood, conventional systems executing processinglike that performed by the system of FIG. 1-0 can be implemented onmultiple threads running on multiple processing cores. Suchparallelization of tasks into multiple thread contexts can provide forincreased throughput. Processor architectures such as MIPS may includedeep instruction pipelines to improve the number of instructions percycle. Further, the ability to run a multi-threaded programmingenvironment results in enhanced usage of existing processor resources.To further increase parallel execution on the hardware, processorarchitectures may include multiple processor cores. Multi-corearchitectures comprising the same type of cores, referred to ashomogeneous core architectures, provide higher instruction throughput byparallelizing threads or processes across multiple cores. However, insuch homogeneous core architectures, the shared resources, such asmemory, are amortized over a small number of processors. In still otherembodiments, multiple offload or host processors can reside on modulesconnected to individual rack units or blades that in turn reside onracks or individual servers. These can be further grouped into clustersand datacenters, which can be spatially located in the same building, inthe same city, or even in different countries. Any grouping level can beconnected to each other, and/or connected to public or private cloudinternets.

In such conventional approaches, memory and I/O accesses can incur ahigh amount of processor overhead. Further, as noted herein, contextswitches in conventional general purpose processing units can becomputationally intensive. It is therefore desirable to reduce contextswitch overhead in a networked computing resource handling a pluralityof networked applications in order to increase processor throughput.Conventional server loads can require complex transport, high memorybandwidth, extreme amounts of data bandwidth (randomly accessed,parallelized, and highly available), but often with light touchprocessing: HTML, video, packet-level services, security, and analytics.Further, idle processors still consume more than 50% of their peak powerconsumption.

In contrast, in an embodiment such as that shown in FIG. 1-0, or anequivalent, complex transport, data bandwidth intensive, frequent randomaccess oriented, “light touch” processing loads can be handled behind asocket abstraction created on multiple offload processor 118 cores. Atthe same time, “heavy touch”, computing intensive loads can be handledby a socket abstraction on a host processor 110 core (e.g., x86processor cores). Such software sockets can allow for a naturalpartitioning of these loads between light touch (e.g., ARM) and heavytouch (e.g., x86) processor cores. By usage of new application levelsockets, according to embodiments, server loads can be broken up acrossthe offload processors 118 and the host processor(s) 110.

To better understand operations of the embodiments disclosed herein,conventional cache schemes are described with reference to FIGS. 1-1 and1-2. Modern operating systems that implement virtual memory areresponsible for the allocation of both virtual and physical memory forprocesses, resulting in virtual to physical translations that occur whena process executes and accesses virtually addressed memory. In themanagement of memory for a process, there is typically no coordinationbetween the allocation of a virtual address range and the correspondingphysical addresses that will be mapped by the virtual addresses. Thislack of coordination can affect both processor cache overhead andeffectiveness when a process is executing.

In conventional systems, a processor allocates memory pages that arecontiguous in virtual memory for each process that is executing. Theprocessor also allocates pages in physical memory, which are notnecessarily contiguous. A translation scheme is established between thetwo schemes of addressing to ensure that the abstraction of virtualmemory is correctly supported by physical memory pages. A processor canemploy cache blocks that are resident close to the processor to meet theimmediate data processing needs. Conventional caches can be arranged ina hierarchy. Level One (L1) caches are closest to the processor,followed by L2, L3, and so on. L2 acts a backup to L1 and so on. Forcaches that are indexed by a part of the process's physical addresses,the lack of correlation between the allocation of virtual and physicalmemory for a range of addresses beyond the size of a memory managementunit (MMU) page results in haphazard and inefficient effects in theprocessor caches. This increases cache overheads and delay is introducedduring a context switch operation.

In physically addressed caches, the cache entry for the next page in thevirtual memory may not correspond to the next contiguous page in thecache—thus degrading the overall performance that can be achieved. Forexample, in FIG. 1-1, contiguous pages in virtual memory 130 (Pages 1and 2 of Process 1) collide in the cache as their physical addresses inphysical memory 132 index to the same location of the physically indexedcache 134 (of the processor). That is, processor cache (i.e., 134) isphysically indexed, and the addresses of the pages in the physicalmemory 132 index to the same page in the processor cache. Furthermore,when the effects of multiple processes accessing a shared cache areconsidered, there is typically a lack of consideration of overall cacheperformance when the OS allocates physical memory to processes. Thislack of consideration results in different processes thrashing in thecache across context switches (e.g., Process 1 and Process 2 in FIG.1-1), which can unnecessarily displace each other's lines, which canresult in an indeterminate number of cache miss/fills upon resuming aprocess, or an increased number of line writebacks across contextswitch.

As described with reference to FIG. 1-2, in other conventionalarrangements, processor caches can alternatively be indexed by a part ofthe process's virtual addresses. Virtually indexed caches are accessedby using a section of the bits of the virtual address of the processor.Pages that are contiguous in virtual memory 130 will be contiguous invirtually indexed caches 136 as seen in FIG. 1-2. As long as processorcaches are virtually indexed, no attention needs to be paid tocoordinating the allocation of physical memory 132 with the allocationof virtual addresses. As programs sweep through virtual address ranges,they will enjoy the benefits of spatial locality in the processor cache.Such set-associative caches can have several entries corresponding to anindex. A given page which maps onto the given cache index can beanywhere in that particular set. Given that there are several positionsavailable for a cache entry, the problems that caused thrashing in thecache across context switches (i.e., as shown in FIG. 1-1) arealleviated to a certain extent with set-associative caches, as theprocessor can afford to keep used entries in the cache to the longestextent possible. For this, caches employ a least recently usedalgorithm. This results in mitigation of some of the problems associatedwith a virtual addressing scheme followed by an operating system, butplaces constraints on the size of the cache. Consequently, bigger,multi-way associative caches can be required to ensure that recentlyused entries are not invalidated/flushed out. The comparator circuitryfor a multi-way set associative cache can be complex to accommodate forparallel comparison, which increases the circuit level complexityassociated with the cache.

A cache control scheme known as “page coloring” has been used by someconventional operating systems to deal with the problem of cache-missesdue to a virtual addressing scheme. If the processor cache wasphysically indexed, the operating system was constrained to look forphysical memory locations that would not index to locations in the cacheof the same color. Under such a cache control scheme, an operatingsystem would have to assess, for every virtual address, those pages inthe physical memory that are allowable based on the index they hash toin the physically indexed cache. Several physical addresses aredisallowed as the indices derived might be of the same color. So, forphysically indexed caches, every page in the virtual memory would becolored to identify its corresponding cache location and determine ifthe next page is allocated to a physical memory and thus a cachelocation of the same color or not. This process would be repeated forevery page, which can be a cumbersome operation. While it improves cacheefficiency, page coloring increases the overhead on the memorymanagement and translation unit as colors of every page would have to beidentified to prevent recently used pages from being overwritten. Thelevel of complexity of the operating system increases correspondingly,as it needs to maintain an indicator of the color of the previousvirtual memory page in the cache.

The problem with a virtually indexed cache is that despite the fact thatthe cache access latencies are higher, there is the pervasive problem ofaliasing. In the case of aliasing, multiple virtual addresses (withdifferent indices) mapping to the same page in the physical memory areat different locations in the cache (due to the different indices). Pagecoloring allows the virtual pages and physical pages to have the samecolor and therefore occupy the same set in the cache. Page coloringmakes aliases to share the same superset bits and index to the samelines in the cache. This removes the problem of aliasing. Page coloringalso imposes constraints on memory allocation. When a new physical pageis allocated on a page fault, a memory management algorithm must pick apage with the same color as the virtual color from the free list.Because systems allocate virtual space systematically, the pages ofdifferent programs tend to have the same colors, and thus some physicalcolors may be more frequent than others. Thus page coloring may impactthe page fault rate. Moreover, the predominance of some physical colorsmay create mapping conflicts between programs in a second-level cacheaccessed with physical addresses. Thus, a processor is faced with a verybig problem with the conventional page coloring scheme just described.Each of the virtual pages could be occupying different pages in thephysical memory such that they occupy different cache colors, but theprocessor would need to store the address translation of each and everypage. Given that a process could be sufficiently large, and each processwould comprise of several virtual pages, the page coloring algorithmcould become very complex. This would also complicate it at the TLB end,as it would need to identify for each page of the processor's virtualmemory, the equivalent physical address. As context switches tend toinvalidate the TLB entries, the processor would need to carry out pagewalks and fill the TLB entries, and this would further add indeterminismand latency to what is a routine context switch.

In this way, in commonly available conventional operating systems,context switches result in collisions in the cache as well as TLB misseswhen a process/thread is resumed. When the process/thread resumes, thereare an indeterminate number of instruction and data cache misses as thethread's working set is reloaded back into the cache (i.e., as thethread resumes in user space and executes instructions, the instructionswill typically have to be loaded into the cache, along with theapplication data). Upon switch-in (i.e., resumption of process/thread),the TLB mappings may be completely or partially invalidated, with thebase of the new thread's page tables written to a register reserved forthat purpose. As the thread executes, the TLB misses will result in pagetable walks (either by hardware or software) which result in TLB fills.Each of these TLB misses has its own hardware costs, including pipelinestall due to an exception (e.g., the overhead created by memory accesseswhen performing a page table walk, along with the associated cachemisses/memory loads if the page tables are not in the cache). Thesecosts are dependent upon what took place in the processor betweensuccessive runs of a process and are therefore not fixed costs.Furthermore, these extra latencies add to the cost of a context switchand detract from the effective execution of a process. As will beappreciated, such foregoing described cache control methods arenon-deterministic with respect to processing time, memory requirements,or other operating system controlled resources, reducing overallefficiency of system operation.

FIG. 1-3 shows a cache control system according to an embodiment. In thecache control system, session contents can be contiguous in a physicallyindexed cache 134′. The described embodiment can use a translationscheme such that contiguous pages of a session in virtual memory 130 arephysically contiguous in the physically indexed cache 134. In contrastto the foregoing described non-deterministic cache control schemes, atleast the duration of a context switch operation can be deterministic.In the described embodiment, replacing the context of a previous processby the context of a new process involves transferring the new processcontext from an external low latency memory such as provided by contextmemory 120 of FIG. 1-0. In the process of context switching, access of amain memory of a system can be avoided (where such accesses can be delayintensive). The process context is prefetched from the context memory120 (which can be a low latency memory). If needed for another contextswitch, process context can be saved once again to the context memory120. In this way, deterministic context switching is achieved, as acontext switch operation can be defined in terms of the number of cyclesand the operations needed to be carried out. Further, use of a lowlatency memory to store context data can provide for rapid contextswitching.

FIGS. 2-0 to 2-5 describe aspects of hardware embodiments of a modulethat can include context switching as described herein. In particularembodiments, such processing modules can include DIMM mountable modules.

FIG. 2-0 is a block diagram of a processing module 200 according to oneembodiment. A processing module 200 can include a physical connector202, a memory interface 204, arbiter logic 206, offload processor(s)208, local memory 210, and control logic 212. A connector 202 canprovide a physical connection to system memory bus. This is in contrastto a host processor which can access a system memory bus via a memorycontroller, or the like. In very particular embodiments, a connector 202can be compatible with a dual in-line memory module (DIMM) slot of acomputing system. Accordingly, a system including multiple DIMM slotscan be populated with one or more processing modules 200, or a mix ofprocessing modules and DIMM modules.

A memory interface 204 can detect data transfers on a system memory bus,and in appropriate cases, enable write data to be stored in theprocessing module 200 and/or read data to be read out from theprocessing module 200. Such data transfers can include the receipt ofpacket data having a particular network identifier. In some embodiments,a memory interface 204 can be a slave interface, thus data transfers arecontrolled by a master device separate from the processing module 200.In very particular embodiments, a memory interface 204 can be a directmemory access (DMA) slave, to accommodate DMA transfers over a systemmemory bus initiated by a DMA master. In some embodiments, a DMA mastercan be a device different from a host processor. In such configurations,processing module 200 can receive data for processing (e.g., DMA write),and transfer processed data out (e.g., DMA read) without consuming hostprocessor resources.

Arbiter logic 206 can arbitrate between conflicting accesses of datawithin processing module 200. In some embodiments, arbiter logic 206 canarbitrate between accesses by offload processor 208 and accessesexternal to the processor module 200. It is understood that a processingmodule 200 can include multiple locations that are operated on at thesame time. It is understood that accesses arbitrated by arbiter logic206 can include accesses to physical system memory space occupied by theprocessor module 200, as well as accesses to other resources (e.g.,cache memory of offload or host processor). Accordingly, arbitrationrules for arbiter logic 206 can vary according to application. In someembodiments, such arbitration rules are fixed for a given processormodule 200. In such cases, different applications can be accommodated byswitching out different processing modules. However, in alternateembodiments, such arbitration rules can be configurable.

Offload processor 208 can include one or more processors that canoperate on data transferred over the system memory bus. In someembodiments, offload processors can run a general operating system orserver applications such as Apache (as but one very particular example),enabling processor contexts to be saved and retrieved. Computing tasksexecuted by offload processor 208 can be handled by the hardwarescheduler. Offload processors 208 can operate on data buffered in theprocessor module 200. In addition or alternatively, offload processors208 can access data stored elsewhere in a system memory space. In someembodiments, offload processors 208 can include a cache memoryconfigured to store context information. An offload processor 208 caninclude multiple cores or one core.

A processor module 200 can be included in a system having a hostprocessor (not shown). In some embodiments, offload processors 208 canbe a different type of processor as compared to the host processor. Inparticular embodiments, offload processors 208 can consume less powerand/or have less computing power than a host processor. In veryparticular embodiments, offload processors 208 can be “wimpy” coreprocessors, while a host processor can be a “brawny” core processor.However, in alternate embodiments, offload processors 208 can haveequivalent computing power to any host processor. In very particularembodiments, a host processor can be an x86 type processor, while anoffload processor 208 can include an ARM, ARC, Tensilica, MIPS,Strong/ARM, or RISC type processor, as but a few examples.

Local memory 210 can be connected to offload processor 208 to enable thestoring of context information. Accordingly, an offload processor 208can store current context information, and then switch to a newcomputing task, then subsequently retrieve the context information toresume the prior task. In very particular embodiments, local memory 210can be a low latency memory with respect to other memories in a system.In some embodiments, storing of context information can include copyingan offload processor 208 cache.

In some embodiments, a same space within local memory 210 is accessibleby multiple offload processors 208 of the same type. In this way, acontext stored by one offload processor can be resumed by a differentoffload processor.

Control logic 212 can control processing tasks executed by offloadprocessor(s). In some embodiments, control logic 212 can be considered ahardware scheduler that can be conceptualized as including a dataevaluator 214, scheduler 216 and a switch controller 218. A dataevaluator 214 can extract “metadata” from write data transferred over asystem memory bus. “Metadata”, as used herein, can be any informationembedded at one or more predetermined locations of a block of write datathat indicates processing to be performed on all or a portion of theblock of write data and/or indicate a particular task/process to whichthe data belongs (e.g., classification data). In some embodiments,metadata can be data that indicates a higher level organization for theblock of write data. As but one very particular embodiment, metadata canbe header information of one or more network packets (which may or maynot be encapsulated within a higher layer packet structure).

A scheduler 216 (e.g., a hardware scheduler) can order computing tasksfor offload processor(s) 208. In some embodiments, scheduler 216 cangenerate a schedule that is continually updated as write data forprocessing is received. In very particular embodiments, a scheduler 216can generate such a schedule based on the ability to switch contexts ofoffload processor(s) 208. In this way, on-module computing prioritiescan be adjusted on the fly. In very particular embodiments, a scheduler216 can assign a portion of physical address space (e.g., memorylocations within local memory 210) to an offload processor 208,according to computing tasks. The offload processor 208 can then switchbetween such different spaces, saving context information prior to eachswitch, and subsequently restoring context information when returning tothe memory space.

Switch controller 218 can control computing operations of offloadprocessor(s) 208. In particular embodiments, according to scheduler 216,switch controller 218 can order offload processor(s) 208 to switchcontexts. It is understood that a context switch operation can be an“atomic” operation, executed in response to a single command from switchcontroller 218. In addition or alternatively, a switch controller 218can issue an instruction set that stores current context information,recalls context information, etc.

In some embodiments, processor module 200 can include a buffer memory(not shown). A buffer memory can store received write data on board theprocessor module. A buffer memory can be implemented on an entirelydifferent set of memory devices, or can be a memory embedded with logicand/or the offload processor. In the latter case, arbiter logic 206 canarbitrate access to the buffer memory. In some embodiments, a buffermemory can correspond to a portion of a system physical memory space.The remaining portion of the system memory space can correspond to otherlike processor modules and/or memory modules connected to the samesystem memory bus. In some embodiments buffer memory can be differentthan local memory 210. For example, buffer memory can have a sloweraccess time than local memory 210. However, in other embodiments, buffermemory and local memory can be implemented with like memory devices.

In very particular embodiments, write data for processing can have anexpected maximum flow rate. A processor module 200 can be configured tooperate on such data at, or faster than, such a flow rate. In this way,a master device (not shown) can write data to a processor module withoutdanger of overwriting data “in process”.

The various computing elements of a processor module 200 can beimplemented as one or more integrated circuit devices (ICs). It isunderstood that the various components shown in FIG. 2-0 can be formedin the same or different ICs. For example, control logic 212, memoryinterface 214, and/or arbiter logic 206 can be implemented on one ormore logic ICs, while offload processor(s) 208 and local memory 210 areseparate ICs. Logic ICs can be fixed logic (e.g., application specificICs), programmable logic (e.g., field programmable gate arrays, FPGAs),or combinations thereof.

Advantageously, the foregoing hardware and systems can provide improvedcomputational performance as compared to traditional computing systems.Conventional systems, including those based on x86 processors, are oftenill-equipped to handle such high volume applications. Even idling, x86processors use a significant amount of power, and near continuousoperation for high bandwidth packet analysis or other high volumeprocessing tasks makes the processor energy costs one of the dominantprice factors.

In addition, conventional systems can have issues with the high cost ofcontext switching wherein a host processor is required to executeinstructions which can include switching from one thread to another.Such a switch can require storing and recalling the context for thethread. If such context data is resident in a host cache memory, such acontext switch can occur relatively quickly. However, if such contextdata is no longer in cache memory (i.e., a cache miss), the data must berecalled from system memory, which can incur a multi-cycle latency.Continuous cache misses during context switching can adversely impactsystem performance.

FIG. 2-1 shows a processor module 200-1 according to one very particularembodiment which is capable of reducing issues associated with highvolume processing or context switching associated with many conventionalserver systems. A processor module 200-1 can include ICs 220-0/1 mountedto a printed circuit board (PCB) type substrate 222. PCB type substrate222 can include in-line module connector 202, which in one veryparticular embodiment, can be a DIMM compatible connector. IC 220-0 canbe a system-on-chip (SoC) type device, integrating multiple functions.In the very particular embodiment shown, an IC 220-0 can includeembedded processor(s), logic and memory. Such embedded processor(s) canbe offload processor(s) 208 as described herein, or equivalents. Suchlogic can be any of controller logic 212, memory interface 204 and/orarbiter logic 206, as described herein, or equivalents. Such memory canbe any of local memory 210, cache memory for offload processor(s) 208,or buffer memory, as described herein, or equivalents. Logic IC 220-1can provide logic functions not included IC 220-0.

FIG. 2-2 shows a processor module 200-2 according to another veryparticular embodiment. A processor module 200-2 can include ICs 220-2,-3, -4, -5 mounted to a PCB type substrate 222, like that of FIG. 2-1.However, unlike FIG. 2-1, processor module functions are distributedamong single purpose type ICs. IC 220-2 can be a processor IC, which canbe an offload processor 208. IC 220-3 can be a memory IC which caninclude local memory 210, buffer memory, or combinations thereof. IC220-4 can be a logic IC which can include control logic 212, and in onevery particular embodiment, can be an FPGA. IC 220-5 can be anotherlogic IC which can include memory interface 204 and arbiter logic 206,and in one very particular embodiment, can also be an FPGA.

It is understood that FIGS. 2-1/2 represent but two of variousimplementations. The various functions of a processor module can bedistributed over any suitable number of ICs, including a single SoC typeIC.

FIG. 2-3 shows an opposing side of a processor module 200-1 or 200-2according to a very particular embodiment. Processor module 200-3 caninclude a number of memory ICs, one shown as 220-6, mounted to a PCBtype substrate 222, like that of FIG. 2-1. It is understood that variousprocessing and logic components can be mounted on an opposing side tothat shown. A memory IC 220-6 can be configured to represent a portionof the physical memory space of a system. Memory ICs 220-6 can performany or all of the following functions: operate independently of otherprocessor module components, providing system memory accessed in aconventional fashion; serve as buffer memory, storing write data thatcan be processed with other processor module components, or serve aslocal memory for storing processor context information.

FIG. 2-4 shows a conventional DIMM module (i.e., it serves only a memoryfunction) that can populate a memory bus along with processor modules asdescribed herein, or equivalents.

FIG. 2-5 shows a system 230 according to one embodiment. A system 230can include a system memory bus 228 accessible via multiple in-linemodule slots (one shown as 226). According to embodiments, any or all ofthe slots 226 can be occupied by a processor module 200 as describedherein, or an equivalent. In the event all slots 226 are not occupied bya processor module 200, available slots can be occupied by conventionalin-line memory modules 224. In a very particular embodiment, slots 226can be DIMM slots.

In some embodiments, a processor module 200 can occupy one slot.However, in other embodiments, a processor module can occupy multipleslots.

In some embodiments, a system memory bus 228 can be further interfacedwith one or more host processors and/or input/output device (not shown).

Having described processor modules according to various embodiments,operations of an offload processor module capable of interfacing withserver or similar system via a memory bus and according to a particularembodiment will now be described.

FIG. 3 shows a system 301 that can execute context switches in offloadprocessors according to an embodiment. In the example shown, a system301 can transport packet data to one or more computational units (oneshown as 300) located on a module, which in particular embodiments, caninclude a connector compatible with an existing memory module. In someembodiments, a computational unit 300 can include a processor module asdescribed in embodiments herein, or an equivalent. A computational unit300 can be capable of intercepting or otherwise accessing packets sentover a memory bus 316 and carrying out processing on such packets,including but not limited to termination or metadata processing. Asystem memory bus 316 can be a system memory bus like those describedherein, or equivalents (e.g., 228).

Referring still to FIG. 3, a system 301 can include an I/O device 302which can receive packet or other I/O data from an external source. Insome embodiments I/O device 302 can include physical or virtualfunctions generated by the physical device to receive a packet or otherI/O data from the network or another computer or virtual machine. In thevery particular embodiment shown, an I/O device 302 can include anetwork interface card (NIC) having input buffer 302 a (e.g., DMA ringbuffer) and an I/O virtualization function 302 b.

According to embodiments, an I/O device 302 can write a descriptorincluding details of the necessary memory operation for the packet (i.e.read/write, source/destination). Such a descriptor can be assigned avirtual memory location (e.g., by an operating system of the system301). I/O device 302 then communicates with an input output memorymanagement unit (IOMMU) 304 which can translate virtual addresses tocorresponding physical addresses with an IOMMU function 304 b. In theparticular embodiment shown, a translation look-aside buffer (TLB) 304 acan be used for such translation. Virtual function reads or writes databetween I/O device and system memory locations can then be executed witha direct memory transfer (e.g., DMA) via a memory controller 306 b ofthe system 301. An I/O device 302 can be connected to IOMMU 304 by ahost bus 312. In one very particular embodiment, a host bus 312 can be aperipheral interconnect (PCI) type bus. IOMMU 304 can be connected to ahost processing section 306 at a central processing unit I/O (CPUI0) 306a. In the embodiment shown, such a connection 314 can support aHyperTransport (HT) protocol.

In the embodiment shown, a host processing section 306 can include theCPUIO 306 a, memory controller 306 b, processing core 306 c andcorresponding provisioning agent 306 d.

In particular embodiments, a computational unit 300 can interface withthe system bus 316 via standard in-line module connection, which in veryparticular embodiments can include a DIMM type slot. In the embodimentshown, a memory bus 316 can be a DDR3 type memory bus. Alternateembodiments can include any suitable system memory bus. Packet data canbe sent by memory controller 306 b via memory bus 316 to a DMA slaveinterface 310 a. DMA slave interface 310 a can be adapted to receiveencapsulated read/write instructions from a DMA write over the memorybus 316.

A hardware scheduler (308 b/c/d/e/h) can perform traffic management onincoming packets by categorizing them according to flow using sessionmetadata. Packets can be queued for output in an onboard memory (310b/308 a/308 m) based on session priority. When the hardware schedulerdetermines that a packet for a particular session is ready to beprocessed by the offload processor 308 i, the onboard memory is signaledfor a context switch to that session. Utilizing this method ofprioritization, context switching overhead can be reduced, as comparedto conventional approaches. That is, a hardware scheduler can handlecontext switching decisions and thus optimize the performance of thedownstream resource (e.g., offload processor 308 i).

As noted above, in very particular embodiments, an offload processor 308i can be a “wimpy core” type processor. According to some embodiments, ahost processor 306 c can be a “brawny core” type processor (e.g., an x86or any other processor capable of handling “heavy touch” computationaloperations). While an I/O device 302 can be configured to trigger hostprocessor interrupts in response to incoming packets, according toembodiments, such interrupts can be disabled, thereby reducingprocessing overhead for the host processor 306 c. In some veryparticular embodiments, an offload processor 308 i can include an ARM,ARC, Tensilica, MIPS, Strong/ARM or any other processor capable ofhandling “light touch” operations. Preferably, an offload processor canrun a general purpose operating system for executing a plurality ofsessions, which can be optimized to work in conjunction with thehardware scheduler in order to reduce context switching overhead.

Referring still to FIG. 3, in operation, a system 301 can receivepackets from an external network over a network interface. The packetsare destined for either a host processor 306 c or an offload processor308 i based on the classification logic and schematics employed by I/Odevice 302. In particular embodiments, I/O device 302 can operate as avirtualized NIC, with packets for a particular logical network or to acertain virtual MAC (VMAC) address can be directed into separate queuesand sent over to the destination logical entity. Such an arrangement cantransfer packets to different entities. In some embodiments, each suchentity can have a virtual driver, a virtual device model that it uses tocommunicate with connected virtual network.

According to embodiments, multiple devices can be used to redirecttraffic to specific memory addresses. So, each of the network devicesoperates as if it is transferring the packets to the memory location ofa logical entity. However, in reality, such packets are transferred tomemory addresses where they can be handled by one or more offloadprocessors (e.g., 308 i). In particular embodiments such transfers areto physical memory addresses, thus logical entities can be removed fromthe processing, and a host processor can be free from such packethandling.

Accordingly, embodiments can be conceptualized as providing a memory“black box” to which specific network data can be fed. Such a memoryblack box can handle the data (e.g., process it) and respond back whensuch data is requested.

Referring still to FIG. 3, according to some embodiments, I/O device 302can receive data packets from a network or from a computing device. Thedata packets can have certain characteristics, including transportprotocol number, source and destination port numbers, source anddestination IP addresses, for example. The data packets can further havemetadata that is processed (308 d) that helps in their classificationand management.

I/O device 302 can include, but is not limited to, peripheral componentinterconnect (PCI) and/or PCI express (PCIe) devices connecting with ahost motherboard via PCI or PCIe bus (e.g., 312). Examples of I/Odevices include a network interface controller (NIC), a host busadapter, a converged network adapter, an ATM network interface, etc.

In order to provide for an abstraction scheme that allows multiplelogical entities to access the same I/O device 302, the I/O device maybe virtualized to provide for multiple virtual devices each of which canperform some of the functions of the physical I/O device. The IOvirtualization program (e.g., 302 b) according to an embodiment, canredirect traffic to different memory locations (and thus to differentoffload processors attached to modules on a memory bus). To achievethis, an I/O device 302 (e.g., a network card) may be partitioned intoseveral function parts; including controlling function (CF) supportinginput/output virtualization (IOV) architecture (e.g., single-root IOV)and multiple virtual function (VF) interfaces. Each virtual functioninterface may be provided with resources during runtime for dedicatedusage. Examples of the CF and VF may include the physical function andvirtual functions under schemes such as Single Root I/O Virtualizationor Multi-Root I/O Virtualization architecture. The CF acts as thephysical resources that sets up and manages virtual resources. The CF isalso capable of acting as a full-fledged 10 device. The VF isresponsible for providing an abstraction of a virtual device forcommunication with multiple logical entities/multiple memory regions.

The operating system/the hypervisor/any of the virtual machines/usercode running on a host processor 306 c may be loaded with a devicemodel, a VF driver and a driver for a CF. The device model may be usedto create an emulation of a physical device for the host processor 306 cto recognize each of the multiple VFs that are created. The device modelmay be replicated multiple times to give the impression to a VF driver(a driver that interacts with a virtual IO device) that it isinteracting with a physical device of a particular type.

For example, a certain device module may be used to emulate a networkadapter such as the Intel® Ethernet Converged Network Adapter(CNA)X540-T2, so that the I/O device 302 believes it is interacting with suchan adapter. In such a case, each of the virtual functions may have thecapability to support the functions of the above said CNA, i.e., each ofthe Physical Functions should be able to support such functionality. Thedevice model and the VF driver can be run in either privileged ornon-privileged mode. In some embodiments, there is no restriction withregard to who hosts/runs the code corresponding to the device model andthe VF driver. The code, however, has the capability to create multiplecopies of device model and VF driver so as to enable multiple copies ofsaid I/O interface to be created.

An application or provisioning agent 306 d, as part of anapplication/user level code running in a kernel, may create a virtualI/O address space for each VF, during runtime and allocate part of thephysical address space to it. For example, if an application handlingthe VF driver instructs it to read or write packets from or to memoryaddresses 0xaaaa to 0xffff, the device driver may write I/O descriptorsinto a descriptor queue with a head and tail pointer that are changeddynamically as queue entries are filled. The data structure may be ofanother type as well, including but not limited to a ring structure 302a or hash table.

The VF can read from or write data to the address location pointed to bythe driver. Further, on completing the transfer of data to the addressspace allocated to the driver, interrupts, which are usually triggeredto the host processor to handle said network packets, can be disabled.Allocating a specific I/O space to a device can include allocating saidIO space a specific physical memory space occupied.

In another embodiment, the descriptor may comprise only a writeoperation, if the descriptor is associated with a specific datastructure for handling incoming packets. Further, the descriptor foreach of the entries in the incoming data structure may be constant so asto redirect all data write to a specific memory location. In analternate embodiment, the descriptor for consecutive entries may pointto consecutive entries in memory so as to direct incoming packets toconsecutive memory locations.

Alternatively, said operating system may create a defined physicaladdress space for an application supporting the VF drivers and allocatea virtual memory address space to the application or provisioning agent306 d, thereby creating a mapping for each virtual function between saidvirtual address and a physical address space. Said mapping betweenvirtual memory address space and physical memory space may be stored inIOMMU tables (e.g., a TLB 304 a). The application performing memoryreads or writes may supply virtual addresses to say virtual function,and the host processor OS may allocate a specific part of the physicalmemory location to such an application.

Alternatively, VF may be configured to generate requests such as readand write which may be part of a direct memory access (DMA) read orwrite operation, for example. The virtual addresses is be translated bythe IOMMU 304 to their corresponding physical addresses and the physicaladdresses may be provided to the memory controller for access. That is,the IOMMU 304 may modify the memory requests sourced by the I/O devicesto change the virtual address in the request to a physical address, andthe memory request may be forwarded to the memory controller for memoryaccess. The memory request may be forwarded over a bus 314 that supportsa protocol such as HyperTransport 314. The VF may in such cases carryout a direct memory access by supplying the virtual memory address tothe IOMMU 304.

Alternatively, said application may directly code the physical addressinto the VF descriptors if the VF allows for it. If the VF cannotsupport physical addresses of the form used by the host processor 306 c,an aperture with a hardware size supported by the VF device may be codedinto the descriptor so that the VF is informed of the target hardwareaddress of the device. Data that is transferred to an aperture may bemapped by a translation table to a defined physical address space in thesystem memory. The DMA operations may be initiated by software executedby the processors, programming the I/O devices directly or indirectly toperform the DMA operations.

Referring still to FIG. 3, in particular embodiments, parts ofcomputational unit 300 can be implemented with one or more FPGAs. In thesystem of FIG. 3, computational unit 300 can include FPGA 310 in whichcan be formed a DMA slave device module 310 a and arbiter 310 f. A DMAslave module 310 a can be any device suitable for attachment to a memorybus 316 that can respond to DMA read/write requests. In alternateembodiments, a DMA slave module 310 a can be another interface capableof block data transfers over memory bus 316. The DMA slave module 310 acan be capable of receiving data from a DMA controller (when it performsa read from a ‘memory’ or from a peripheral) or transferring data to aDMA controller (when it performs a write instruction on the DMA slavemodule 310 a). The DMA slave module 310 a may be adapted to receive DMAread and write instructions encapsulated over a memory bus, (e.g., inthe form of a DDR data transmission, such as a packet or data burst), orany other format that can be sent over the corresponding memory bus.

A DMA slave module 310 a can reconstruct the DMA read/write instructionfrom the memory R/W packet. The DMA slave module 310 a may be adapted torespond to these instructions in the form of data reads/data writes tothe DMA master, which could either be housed in a peripheral device, inthe case of a PCIe bus, or a system DMA controller in the case of an ISAbus.

I/O data that is received by the DMA device 310 a can then queued forarbitration. Arbitration can include the process of scheduling packetsof different flows, such that they are provided access to availablebandwidth based on a number of parameters. In general, an arbiter 310 fprovides resource access to one or more requestors. If multiplerequestors request access, an arbiter 310 f can determine whichrequestor becomes the accessor and then passes data from the accessor tothe resource interface, and the downstream resource can begin executionon the data. After the data has been completely transferred to aresource, and the resource has competed execution, the arbiter 310 f cantransfer control to a different requestor and this cycle repeats for allavailable requestors. In the embodiment of FIG. 3 arbiter 310 f cannotify other portions of computational unit 300 (e.g., 308) of incomingdata.

Alternatively, a computation unit 300 can utilize an arbitration schemeshown in U.S. Pat. No. 7,813,283, issued to Dalal on Oct. 12, 2010, thecontents of which are incorporated herein by reference. Other suitablearbitration schemes known in art could be implemented in embodimentsherein. Alternatively, the arbitration scheme of the current inventionmight be implemented using an OpenFlow switch and an OpenFlowcontroller.

In the very particular embodiment of FIG. 3, computational unit 300 canfurther include notify/prefetch circuits 310 c which can prefetch datastored in a buffer memory 310 b in response to DMA slave module 310 a,and as arbitrated by arbiter 310 f. Further, arbiter 310 f can accessother portions of the computational unit 300 via a memory mapped I/Oingress path 310 e and egress path 310 g.

Referring to FIG. 3, a hardware scheduler can include a schedulingcircuit 308 b/n to implement traffic management of incoming packets.Packets from a certain source, relating to a certain traffic class,pertaining to a specific application or flowing to a certain socket arereferred to as part of a session flow and are classified using sessionmetadata. Such classification can be performed by classifier 308 e.

In some embodiments, session metadata 308 d can serve as the criterionby which packets are prioritized and scheduled and as such, incomingpackets can be reordered based on their session metadata. Thisreordering of packets can occur in one or more buffers and can modifythe traffic shape of these flows. The scheduling discipline chosen forthis prioritization, or traffic management (TM), can affect the trafficshape of flows and micro-flows through delay (buffering), bursting oftraffic (buffering and bursting), smoothing of traffic (buffering andrate-limiting flows), dropping traffic (choosing data to discard so asto avoid exhausting the buffer), delay jitter (temporally shifting cellsof a flow by different amounts) and by not admitting a connection (e.g.,cannot simultaneously guarantee existing service level agreements (SLAs)with an additional flow's SLA).

According to embodiments, computational unit 300 can serve as part of aswitch fabric, and provide traffic management with depth-limited outputqueues, the access to which is arbitrated by a scheduling circuit 308b/n. Such output queues are managed using a scheduling discipline toprovide traffic management for incoming flows. The session flows queuedin each of these queues can be sent out through an output port to adownstream network element.

It is noted that conventional traffic management do not take intoaccount the handling and management of data by downstream elementsexcept for meeting the SLA agreements it already has with saiddownstream elements.

In contrast, according to embodiments a scheduler circuit 308 b/n canallocate a priority to each of the output queues and carry outreordering of incoming packets to maintain persistence of session flowsin these queues. A scheduler circuit 308 b/n can be used to control thescheduling of each of these persistent sessions into a general purposeoperating system (OS) 308 j, executed on an offload processor 308 i.Packets of a particular session flow, as defined above, can belong to aparticular queue. The scheduler circuit 308 b/n may control theprioritization of these queues such that they are arbitrated forhandling by a general purpose (GP) processing resource (e.g., offloadprocessor 308 i) located downstream. An OS 308 j running on a downstreamprocessor 308 i can allocate execution resources such as processorcycles and memory to a particular queue it is currently handling. The OS308 j may further allocate a thread or a group of threads for thatparticular queue, so that it is handled distinctly by the generalpurpose processing element 308 i as a separate entity. The fact thatthere can be multiple sessions running on a GP processing resource, eachhandling data from a particular session flow resident in a queueestablished by the scheduler circuit, tightly integrates the schedulerand the downstream resource (e.g., 308 i). This can bring aboutpersistence of session information across the traffic management andscheduling circuit and the general purpose processing resource 308 i.

Dedicated computing resources (e.g., 308 i), memory space and sessioncontext information for each of the sessions can provide a way ofhandling, processing and/or terminating each of the session flows at thegeneral purpose processor 308 i. The scheduler circuit 308 b/n canexploit this functionality of the execution resource to queue sessionflows for scheduling downstream. The scheduler circuit 308 b/n can beinformed of the state of the execution resource(s) (e.g., 308 i), thecurrent session that is run on the execution resource; the memory spaceallocated to it, the location of the session context in the processorcache.

According to embodiments, a scheduler circuit 308 b/n can furtherinclude switching circuits to change execution resources from one stateto another. The scheduler circuit 308 b/n can use such a capability toarbitrate between the queues that are ready to be switched into thedownstream execution resource. Further, the downstream executionresource can be optimized to reduce the penalty and overhead associatedwith context switch between resources. This is further exploited by thescheduler circuit 308 b/n to carry out seamless switching betweenqueues, and consequently their execution as different sessions by theexecution resource.

According to embodiments, a scheduler circuit 308 b/n can scheduledifferent sessions on a downstream processing resource, wherein the twoare operated in coordination to reduce the overhead during contextswitches. An important factor in decreasing the latency of services andengineering computational availability can be hardware context switchingsynchronized with network queuing. In embodiments, when a queue isselected by a traffic manager, a pipeline coordinates swapping in of thecache (e.g., L2 cache) of the corresponding resource (e.g., 308 i) andtransfers the reassembled I/O data into the memory space of theexecuting process. In certain cases, no packets are pending in thequeue, but computation is still pending to service previous packets.Once this process makes a memory reference outside of the data swapped,the scheduler circuit (308 b/n) can enable queued data from an I/Odevice 302 to continue scheduling the thread.

In some embodiments, to provide fair queuing to a process not havingdata, a maximum context size can be assumed as data processed. In thisway, a queue can be provisioned as the greater of computational resourceand network bandwidth resource. As but one very particular example, acomputation resource can be an ARM A9 processor running at 800 MHz,while a network bandwidth can be 3 Gbps of bandwidth. Given the lopsidednature of this ratio, embodiments can utilize computation having manyparallel sessions (such that the hardware's prefetching ofsession-specific data offloads a large portion of the host processorload) and having minimal general purpose processing of data.

Accordingly, in some embodiments, a scheduler circuit 308 b/n can beconceptualized as arbitrating, not between outgoing queues at line ratespeeds, but arbitrating between terminated sessions at very high speeds.The stickiness of sessions across a pipeline of stages, including ageneral purpose OS, can be a scheduler circuit optimizing any or allsuch stages of such a pipeline.

Alternatively, a scheduling scheme can be used as shown in U.S. Pat. No.7,760,715 issued to Dalal on Jul. 20, 2010, incorporated herein byreference. This scheme can be useful when it is desirable to rate limitthe flows for preventing the downstream congestion of another resourcespecific to the over-selected flow, or for enforcing service contractsfor particular flows. Embodiments can include arbitration scheme thatallows for service contracts of downstream resources, such as generalpurpose OS that can be enforced seamlessly.

Referring still to FIG. 3, a hardware scheduler according to embodimentsherein, or equivalents, can provide for the classification of incomingpacket data into session flows based on session metadata. It can furtherprovide for traffic management of these flows before they are arbitratedand queued as distinct processing entities on the offload processors.

In some embodiments, offload processors (e.g., 308 i) can be generalpurpose processing units capable of handling packets of differentapplication or transport sessions. Such offload processors can be lowpower processors capable of executing general purpose instructions. Theoffload processors could be any suitable processor, including but notlimited to: ARM, ARC, Tensilica, MIPS, StrongARM or any other processorthat serves the functions described herein. Such offload processors havea general purpose OS running on them, wherein the general purpose OS isoptimized to reduce the penalty associated with context switchingbetween different threads or group of threads.

In contrast, context switches on host processors can be computationallyintensive processes that require the register save area, process contextin the cache and TLB entries to be restored if they are invalidated oroverwritten. Instruction Cache misses in host processing systems canlead to pipeline stalls and data cache misses lead to operation stalland such cache misses reduce processor efficiency and increase processoroverhead.

Also in contrast, an OS 308 j running on the offload processors 308 i inassociation with a scheduler circuit 308 b/n, can operate together toreduce the context switch overhead incurred between different processingentities running on it. Embodiments can include a cooperative mechanismbetween a scheduler circuit and the OS on the offload processor 308 i,wherein the OS sets up session context to be physically contiguous(physically colored allocator for session heap and stack) in the cache;then communicates the session color, size, and starting physical addressto the scheduler circuit upon session initialization. During an actualcontext switch, a scheduler circuit can identify the session context inthe cache by using these parameters and initiate a bulk transfer ofthese contents to an external low latency memory (e.g., 308 g). Inaddition, the scheduler circuit can manage the prefetch of the oldsession if its context was saved to a local memory 308 g. In particularembodiments, a local memory 308 g can be low latency memory, such as areduced latency dynamic random access memory (RLDRAM), as but one veryparticular embodiment. Thus, in embodiments, session context can beidentified distinctly in the cache.

In some embodiments, context size can be limited to ensure fastswitching speeds. In addition or alternatively, embodiments can includea bulk transfer mechanism to transfer out session context to a localmemory 308 g. The cache contents stored therein can then be retrievedand prefetched during context switch back to a previous session.Different context session data can be tagged and/or identified withinthe local memory 308 g for fast retrieval. As noted above, contextstored by one offload processor may be recalled by a different offloadprocessor.

In the very particular embodiment of FIG. 3, multiple offload processingcores can be integrated into a computation FPGA 308. Multiplecomputational FPGAs can be arbitrated by arbitrator circuits in anotherFPGA 310. The combination of computational FPGAs (e.g., 308) and arbiterFPGAs (e.g., 310) are referred to as “XIMM” modules or “Xockets DIMMmodules” (e.g., computation unit 300). In particular applications, theseXIMM modules can provide integrated traffic and thread managementcircuits that broker execution of multiple sessions on the offloadprocessors.

FIG. 3 also shows an offload processor tunnel connection 308 k, as wellas a memory interface 308 m and access unit 308 l (which can be anaccelerator coherency port (ACP)). Memory interface 308 m can accessbuffer memory 308 a. According to embodiments, system 301 can includeuse an access (or “snooping” unit) 308 l, to access the cache contentsof an offload processor 308 i. In particular embodiments, the cacheaccessed can be an L2 cache. An access unit 308 l can provide a port orother access capability that can load data from an external, non-cachedmemory 308 g into an offload processor cache, as well as transfer thecache contents of an offload processor 308 i to a non-cache memory 308g. As part of a computational element 300, there can be several memorydevices (e.g., RAMs) that form memory 308 g. Thus, memory 308 g can beused to store the cache contents of sessions. Memory 308 g can includeone or more low latency memories, and can be conceptualized assupplementing and/or augmenting the available L2 cache, and extendingthe coherency domain of sessions. The addition memory 308 g and accessunit 308 l can reduce the adverse effects of cache misses for switchedin sessions, in that a session's context can be fetched and pre-fetchedinto an offload processor cache so that the when the thread resumes,most of its previous working set is already present in the cache.

According to one particular embodiment, in a session switch-out, anoffload processor 308 i cache contents can be transferred to memory 308g via tunnel 308 k. However, in some embodiments, a thread's registerset may be saved to memory as part of switch-out, thus these registercontents can be resident in the cache. Therefore, as part of switch-in,when a session's contents are prefetched and transferred into the cacheof an offload processor 308 i, the register contents can be loaded bythe kernel upon resuming the thread, and these loads should be from thecache and not from memory 308 g. Thus, with the careful management of asession's cache contents, the cost of context switching due to registerset save and restore and cache misses on switch-in can be greatlyreduced, and even eliminated in some optimal cases, thereby eliminatingtwo sources of context switch overhead and reducing the latency for theswitched-in session to resume useful processing.

According to some embodiments, an access (or snooping) unit (e.g., 308l) can have the indices of all the lines in the cache where the relevantsession context resides. If the session is scattered across locations ina physically indexed cache, it can become very cumbersome to access allof the session contents as multiple address translations would berequired to access multiple pages of the same session. Accordingly,embodiments can include a page coloring scheme, in which the sessioncontents are established in contiguous locations in a physically indexedcache. A memory allocator for session data can allocate from physicallycontiguous pages so that there is control over physical address rangesfor the sessions. In some embodiments, this is done by aligning thevirtual memory page and the physical memory page to index to the samelocation in the cache (e.g., FIG. 1-3). In alternate embodiments,virtual and physical memory pages do not have to index the same locationin a physically indexed cache, but the different pages of the sessioncan be contiguous in physical memory, such that knowledge of thebeginning index and size of the entry in the cache suffices to accessall session data. Further, the set size is equal to the size of asession, so that once the index of a session entry in the cache isknown; the index, the size and the set color could be used to completelytransfer out the session contents from the cache to an external memory(e.g., 308 g).

According to embodiments, all pages of a session can be assigned thesame color in a cache of an offload processor. In a particularembodiment, all pages of a session can start at the page boundary of adefined color. The number of pages allocated to a color can be fixedbased on the size of a session in the cache. An offload processor (e.g.,308 i) can be used for executing a specific type of sessions and it isinformed of the size of each session beforehand. Based on this, theoffload processor can begin a new entry at a session boundary. It cansimilarly allocate pages in physical memory that index to the sessionboundary in the cache. The entire cache context can be saved beginningat the session boundary. In the currently described embodiment, multiplepages in the session can be contiguous in a physically indexed cache.Multiple pages of a session can have the same color (i.e., they are partof the same set) and are located contiguously. Pages of a session areaccessible by using an offset from the base index of the session. Thecache can be arranged and broken up into distinct sets, not as pages,but as sessions. To move from one session to another, the memoryallocation scheme uses an offset to the lowest bit of the indexes usedto access these sessions. For example, a physically indexed cache can bean L2 cache having a size of 512 kb. The cache can be 8-way associative,with eight ways per set possible in the L2 cache. Therefore, there areeight lines per any color in L2, or eight separate instances of eachcolor in L2. With a session context size of 8 Kb, there will then beeight different session areas within the 512 Kb L2 cache, or eightsession colors with these chosen sizes.

According to embodiments, a physical memory allocator can identify thecolor corresponding to a session based on the cache entry/main memoryentry of the temporally previous session. In a particular embodiment,the physical memory allocator can identify a session of the previoussession based on the 3 bits of the address used to assign a cache entryto the previous session. The physical memory allocator can assign thenew session to a main memory location (whose color can be determinedthrough a few comparisons to the most recently used entry) and willcause a cache entry corresponding to a session of a different color tobe evicted based on a least recently used policy. In another embodiment,an offload processor can include multiple cores. In such an embodiment,cache entries can be locked out for use by each processor core. Forexample, if the offload processor has two cores, a given set of cachelines in a cache (i.e., L2 cache) can be divided among processors,halving the number of colors. The color of the session, index of thesession and session size to an external scheduler when a new session iscreated can be communicated. This information can be used for queuemanagement of incoming session flows.

Embodiments can also permit isolation of shared text and any shared databy locking these lines into a cache, apart from session data. Again, aphysical memory allocator and physical coloring techniques can be used.If separate shared data is placed in a cache, it is possible to lock itinto the cache, as long as transfers by an access unit (e.g., ACP) donot copy such lines. When allocating memory for session data, the memoryallocator can be aware of physical color, as session data that residesin the cache is mapped.

Having described various embodiments suitable for cache and contextswitching management operations, an example illustrating particularaspects will now be described.

FIG. 4 shows a method 400 of reduced overhead context switching for asystem according to an embodiment. At initialization, a determinationcan be made if session coloring is required (402). Such a determinationcan be made by an OS. If session coloring is not required (No from 402),page coloring may or may not be present depending upon default choicesof an OS (424).

If session coloring is required (Yes from 402), an OS can initializes amemory allocator (404). A memory allocator can employ cache optimizationtechniques that can allocate each session entry to a “session” boundary.The memory allocator can determine the starting address of each session,the number of sessions allowable in the cache, and the number oflocations wherein a session can be found for a given color. Suchoperations can include determining the number of sets available basedcache size, number of colors and size of a session (step 406).

When a packet for a session arrives, a determination can be made ofwhether the packet is for a current or different session (408). Such anaction can be performed by the OS. If a packet is for a differentsession (Yes from 408), a determination can be made of whether thepacket is form an earlier session (410). If the packet is not from anearlier session (i.e., it is a new session), a determination can be madeto see if there is enough memory for the new session (418). If there isenough space (Yes from 418), a switch can be made to the new session(422). Such an action can include allocating a new session at a sessionboundary, and saving the context of the process that is currentlyexecuting to a context memory (which can be an external low latencymemory).

If no cache memory is available for a new session (No from 418) and/orthe packet is for an earlier session (Yes from 410) an examination canbe made to determine if the packet of the old/new session is of a samecolor (412). If it is of a different color (No from 412), a switch canbe made to that session (step 414). Such an action can includeretrieving (for an earlier session) or creating (for a new session)cache entries for the task. In addition, such an action can include, ifneeded, the flushing of cache entries according to an LRU scheme.

If a packet of the old/new session is of the same color (Yes from 412),a determination can be made as to whether the color pressure can beexceeded (416). If the color pressure can be exceeded, or a session ofsome other color is not available (Yes, or . . . from 416), a switch canbe made to the new session (420). Such an action can include creatingcache entries and remembering the new session color. If the colorpressure cannot be exceeded, but sessions of some other color areavailable (No, but . . . from 416), a method can proceed to 414.

It should be appreciated that in the foregoing description of exemplaryembodiments of the invention, various features of the invention aresometimes grouped together in a single embodiment, figure, ordescription thereof for the purpose of streamlining the disclosureaiding in the understanding of one or more of the various inventiveaspects. This method of disclosure, however, is not to be interpreted asreflecting an intention that the claimed invention requires morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive aspects lie in less than allfeatures of a single foregoing disclosed embodiment. Thus, the claimsfollowing the detailed description are hereby expressly incorporatedinto this detailed description, with each claim standing on its own as aseparate embodiment of this invention.

It is also understood that the embodiments of the invention may bepracticed in the absence of an element and/or step not specificallydisclosed. That is, an inventive feature of the invention may beelimination of an element.

Accordingly, while the various aspects of the particular embodiments setforth herein have been described in detail, the present invention couldbe subject to various changes, substitutions, and alterations withoutdeparting from the spirit and scope of the invention.

What is claimed is:
 1. A method for context switching of multipleoffload processors, comprising the steps of: receiving network packetsfor processing through a memory bus connected socket, organizing thenetwork packets into multiple sessions for processing, suspendingprocessing of at least one session by reading a cache state of at leastone of the offload processor into a context memory by operation of ascheduling circuit, with virtual memory locations and physical cachelocations being aligned, and subsequently directing transfer of thecache state to at least one of the offload processors for processing byoperation of the scheduling circuit.
 2. The method of claim 1 whereinthe bulk read is through an accelerator coherency port.
 3. The method ofclaim 1 wherein the associated cache state includes at least one of: astate of offload processor registers, instructions for execution by theoffload processor, a stack pointer, program counter, prefetchedinstructions for execution by the offload processor, prefetched data foruse by the offload processor, and data written into the cache of theoffload processor.
 4. The method of claim 1, further including: thecache state includes session context; and setting the session context tobe physically contiguous in the cache of the offload processor.
 5. Themethod of claim 4, wherein the setting of the session context includescooperation of an operating system (OS) running on the offload processorand the scheduling circuit.
 6. The method of claim 1, further includingupon initialization of a processing session, communicating session datato the scheduling circuit.
 7. The method of claim 6, wherein the sessiondata includes any selected from: a session color, a session size andstarting physical cache address for the processing session.
 8. Themethod of claim 1, further including determining starting address foreach of a plurality of processing sessions, the number of sessionsallowable in a cache of an offload processor, and the number oflocations wherein a session can be found for a given session color. 9.The method of claim 1 further including transferring the cache state ofone of the offload processors to the cache of another of the offloadprocessors.
 10. The method of claim 1, further including prioritizing aprocessing of network packets in a first queue received over the memorybus by stopping a first session associated with one of the offloadprocessors, storing the cache state of the offload processor, andinitiating processing of network packets held in a second queue.
 11. Themethod of claim 1, wherein the suspending processing of at least onesession includes operating in a preemption mode to control a sessionexecution.
 12. The method of claim 1, wherein receiving network packetsincludes receiving network packets through a dual in line memory module(DIMM) compatible socket.
 13. The method of claim 1, wherein reading thecache state into the context memory includes reading the cache statedata over the memory bus.